Method for capturing photographs and videos on a handheld client device without continually observing the device&#39;s screen

ABSTRACT

A client device for image capture in an eyes-free manner or without ongoing observance of the device&#39;s display screen to ensure proper framing and focusing. The client device may take the form of a smartphone, tablet, or the like. The client device includes an image capture application run by a processor to select a target to be included in a digital image captured by the client device&#39;s camera. The image capture application causes the processor to operate feedback mechanisms to prompt an operator of the client device to move the device so as to move the camera or its lens, before or during the capturing of the digital image, from a first orientation to a second orientation relative to the target. The image capture application urges the operator of the client device to hold their client device so as to better frame or focus on the selected target.

BACKGROUND

1. Field of the Description

The present invention relates, in general, to digital cameras and otherportable devices configured for capturing digital photographs and videossuch as smartphones, tablets, handheld video games, and the like, andmore particularly, to a method and corresponding apparatus for allowingan operator of a digital device to capture a desired shot, image, video,or the like of an event without having to look at a display screen ortouchscreen of their portable digital device.

2. Relevant Background.

Today, with advances in technologies and the lowering of prices, nearlyeveryone uses or at least has access to digital devices that are adaptedto perform numerous tasks. For example, many digital devices aredesigned to perform one main task such as to act as a cellulartelephone, to allow an operator to read books and other publicationsthat are in digital form (e.g., electronic books or ebooks), to accessthe Internet to allow the user to download and watch videos or playgames and access numerous websites, and the like. Further, though, ithas been common for the digital device, such as a smartphone, tablet, orthe like, to also be configured to digitally capture still photographsand/or videos (“digital images”) or, in other cases, the user may use ahandheld device specifically designed to be a digital camera to takesuch digital images. This collection of digital devices may becollectively thought of as or labeled “handheld client devices” or, moresimply, “client devices.”

Each of the handheld client devices typically will in fact be a smallcomputer or computing device with one or more processors managing memory(e.g., to store the digital images) and running an operating system (OS)and a software suite. The software suite includes applications such as abrowser (e.g., a full HTML web browser) to access the Internet and itsvoluminous data, but many other applications or “apps” (or “mobileapps”) may be included to perform other tasks such as to provide mapswith many mobile digital devices including a GPS navigation unit, toplay media such as music and movies, to access email, and so on. Thehandheld client devices typically will include one or more screens,which may also be touchscreens, to display data and images and to allowthe operator of the devices to interact with their devices includinginitiating and using all of their apps. Commonly used handheld clientdevices include smartphones, digital cameras, and a wide variety ofportable or mobile computers adapted for wireless communications (accessto the Internet via WiFi networks) such as tablets, laptops, pads,handheld game consoles and the like.

Handheld client devices have become immensely popular with nearlyeveryone carrying at least one with them at all times. These devices areuseful for making life more enjoyable and for reducing stress such as bymaking communications easy and allowing the user to efficiently findtheir way around a new location or city. However, client devices aspresently designed often also detract from the user enjoying the presentmoment or experience. For example, an observer of a crowd at any liveevent, such as a concert, a parade, a sporting event, a fireworks orlight show, a play, a wedding, and so on, will look into a sea ofcellphone and tablet screens. This is because a large percentage of theaudience is not personally or directly watching and enjoying the liveevent but, instead, are operating their device to film or capture astill photograph or video of the event to share with others (e.g.,quickly post in an online social media service) or to remember theevent. The members of the audience want to capture the special moment,but one downside is that much of the audience does not capture it withtheir own eyes but mediated by or through the screen of their clientdevice.

Hence, there remains a need for methods and systems to assist operatorsof digital devices or handheld client devices such as smartphones andmobile computers to better capture a photograph and a video (i.e., adigital image(s)). Preferably, the methods and systems would be adaptedto allow the user to confidently capture the “perfect shot” of a liveevent they are attending or participating in without actually needing tolook (or continually look) through the screen of their device (e.g., inan “eyes-free manner”).

SUMMARY

Briefly, a solution is described herein to allow an operator of ahandheld client device to effectively capture digital images such as avideo of a targeted item (e.g., a portion of a live event). The solutionis configured to capture the digital images, from a desired angle inmany cases, without requiring the user to use their screen to manuallyfocus and aim their device's camera lens at the targeted item, e.g.,without having to continually look through the device's screen duringall or most of a captured video.

One embodiment includes a client device with a digital image capturemodule (e.g., a mobile application or app), which may be anOS-appropriate, native application downloaded onto or installed on theclient device. The digital image capture module or image capture app isconfigured to use the client device's embedded sensors to enableeyes-free photo and video capture. For example, an event venue may beused to provide a live event or show such as an amusement or theme parkproviding a parade along a predefined parade route, a light or fireworkshow over a lagoon, or a concert in a particular show arena. In suchsettings, the show or live event takes place at a known location inthree dimensional (3D) space, and this event/show location may be storedin the device's memory (or retrieved in real time) for later use inimage capture operations.

In response to a user/operator initiating an image capturesequence/process, the digital image capture module may access acollection or set of device sensor data to better capture a digitalimage such as by using the client device's embedded global positioningsystem (GPS) sensor to establish location, using the device's compass toestablish azimuth, and using the device's accelerometer to establishpitch. The module processes this data in combination with the knownlocation of the show/live event (which may be retrieved by the modulevia wireless accessing of a communications network such as the Internetor may be stored in the device's memory when the module/mobile app isdownloaded or at a later time) to determine when the device's camera (orthe camera's lens) is properly oriented towards the show or live event(e.g., determine when the camera's lens is effectively targeting theaction).

Other features of the client device may be operated by the image capturemodule (or via communications with the OS running such devices) toprovide the user feedback to direct them how to best position or orienttheir device and its camera lens to best capture the desired digitalimage. For example, the device's rumble feature or vibrate elements maybe used to provide feedback to indicate to the user holding the devicehow they need to adjust the orientation of the device to continue toframe the action/live event. In practice or operation of the imagecapture module, the user may simply hold up their device in aquasi-arbitrary manner to generally aim or focus on the desired target(e.g., generally aim their device's camera lens at a parade, a bandplaying a concert, a light show in the sky, and so on), enjoy the liveevent with their own eyes (without staring at the device's smallscreen), and still capture the digital image recording the moment forposterity (e.g., with eyes-free movement of the device in response tofeedback from the device's rumble/vibrate or other components (such asan audio feedback mechanism providing noises or spoken instructionsregarding proper orientation)).

When launched, the image capture module or app may be adapted to promptthe user by generating an image and/or text on the device's screen toselect a still image or video image mode. Next, the image capture moduleor app may update the screen display to ask the user to select a show orevent (or even a physical structure) they would like to capture orrecord. In some cases, a menu that is context sensitive may be displayedbased on the present geographical location of the client device. Inother cases, the “selection” of the show or event will be automated bythe image capture module or app such as based upon the device's presentlocation and the time of day (e.g., only one event or show is scheduledfor a venue or near a physical location at that time of day such thatthe module or app can determine which show is to be recorded with astill or video image). Once an event or show or other targeted item isselected, the image capture module or app may function to prompt theuser to orient their device (and its camera lens) toward the targeteditem (e.g., at a show start time or at a start time entered by the user)and to operate the device's camera to capture the still or video image(at a show or event predefined length or by a record duration timeentered by the user). The user can enjoy the event or show withoutdistractions associated with glancing back and forth at the device'sscreen.

More particularly, a client device is provided that is specially adaptedfor enhanced image capture such as in an eyes-free manner. The clientdevice, which may take the form of a smartphone, tablet, or the like,includes a processor, memory accessible by the processor, a camera witha lens, and a body containing the processor, the memory, and the camera.In practice, the camera operates to capture or record a digital imagevia the lens and to store the captured digital image in the memory.Significantly, the client device also includes an image captureapplication run by the processor to select a target item to be includedin the captured digital image.

During use of the client device (e.g., during an image capture process),the image capture application causes the processor to operate one ormore feedback mechanisms to prompt an operator of the client device tomove the body so as to move the lens of the camera, before or during thecapturing of the digital image, from a first orientation to a secondorientation relative to the target item. In this manner, the imagecapture application assists or urges the operator of the client deviceto hold their client device so as to better target and/or focus on theselected target item such as a stage on which a live show is occurring,a street upon which a parade is passing, a section of the sky in which afireworks, water, or light show is presented, a field on which asporting event is being played, and so on.

In some implementations of the client device, the image captureapplication determines the first orientation and the second orientationusing orientation sensing components in the client device (e.g., acompass, an accelerometer, and the like). For example, the firstorientation may be defined by a first pitch and a first azimuth and thesecond orientation may be defined by a second pitch and a secondazimuth. In such cases, the image capture application further can act todetermine a present location of the client device, and the secondorientation is selected by the image capture application so as to bewithin a range of acceptable orientations for capturing an image of thetarget item from the present location.

In some implementations, the image capture application further acts todetermine a present location of the client device, to determine a firstheight of the lens of the camera, to compare the first height of thecamera to an acceptable range of heights for capturing an image of thetarget item at the present location of the client device, and when thefirst height is outside of the acceptable range of heights to operatethe one or more feedback mechanisms to prompt the operator to move thecamera from the first height to a second height within the acceptablerange of heights. In the same or other cases, the image captureapplication further acts to determine a present location of the clientdevice. Then, the second orientation is selected to provide the camerawith a field of view of the target item from the present location.

The feedback mechanisms may include a vibration mechanism selectivelyoperable until the lens of the camera is in the second orientation(e.g., rumble the body when out of proper focus to encourage theoperator to move the client device). In some cases, though, the feedbackmechanisms include a display screen on a face of the body, and thedisplay screen is operated to change from a first color to a secondcolor when the lens of the camera is in the second orientation. In othersituations, the feedback mechanisms include a mechanism adapted to movethe body of the client device within a hand of the operator toautomatically move the lens of the camera from the first orientation tothe second orientation.

It may be useful in some client devices for the target item to beselected by the image capture application based on a determination of apresent location of the client device. In other cases, though, the imagecapture application acts to determine a present location of the clientdevice, to lookup a set of potential targets local to the presentlocation, and to display the set of potential targets on a displayscreen of the client device to prompt the operator to indicate thetarget item from the set of potential targets.

In some cases, the client device is adapted such that the image captureapplication responds to user input to augment the captured digital imageusing augmentation layers. These layers/image files may be selected ormodified based on at least one of an orientation of the lens of thecamera relative to the target item, a distance between the lens of thecamera to the target item, and a height of the lens of the camera duringthe capturing of the digital image. In this way, the informationcollected as part of assisting the user in using their device to captureimages in an eyes-free manner are later used to select and/oredit/modify augmentation images so that these augmentation layers/imagesappear to have been collected or captured from the digital camera at thesame location, height, distance, and/or orientation as the originallycaptured image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a functional block diagram of a client device, such as asmartphone or tablet, configured according to one embodiment foreyes-free image capture;

FIG. 2 is a flow diagram of an image capture method (or algorithm)carried out by or via a client device with image capture software or amobile app that causes the device's processor to act as a specialpurpose computer/device; and

FIGS. 3-6 illustrate a client device that is configured according to thepresent description with image capture software (or a mobile app) andthat is being used by an operator or user to capture or record a digitalimage of a user-selected target (or targeted item) in a guided buteyes-free manner.

DETAILED DESCRIPTION

FIG. 1 illustrates a client device 110, with a functional block orschematic diagram, that is adapted specially to allow its user tocapture an image (i.e., digital still or video image 180 in devicememory 160) in an eyes-free manner or without continually looking at thedevice screen 152 rather than watching the live event (or “targeteditem” for a photo or video). The device 110 may take a wide variety offorms but typically will be a portable or handheld electronic orcomputer device that is configured for taking still and/or video images,and the user of the device 110 typically will carry it with them to alocation such as a show venue, a sports stadium, an amusement park, orthe like where live entertainment or other targeted items for imagecapture are present. For example, the client device may take the form ofa cellular phone (e.g., a smartphone), a mobile computer such as atablet, pad, laptop, handheld, or the like, a videogame handheld device,a digital reader, a digital camera, or other electronic/computingdevice.

The client device 110 includes a processor(s) 112 that runs and/or usesan operating system (OS) 120 to run or execute a software suite ornumber of applications (e.g., mobile apps performing one or morefunctions). Particularly, the device 110 includes code in the form of animage capture module or app 130 that is appropriate for the OS 120, andthe image capture module 130 causes the device 110 to perform thefunctions described herein to capture one or more still or video images180 in the device's memory 160 managed by the processor 112. To thisend, the device 110 also includes a digital camera 114 that isselectively operable by the image capture module 130 via the processor112 (and a camera driver/operation software, not shown but understood bythose skilled in the art) to focus upon and capture/record digitalimages 180 of elements viewable through the camera lens (or lenses suchas forward looking lenses and rearward lenses) 116. Briefly, the module130 acts to assist an operator or user of the client device 110 toorient the device such that its camera 114 or, more specifically, thecamera lens 116 is oriented properly to be directed toward a targeteditem such as a stage of a live show, upward into the sky at a desiredangle of a light or fireworks display, or toward a parade passing alonga predefined parade route.

The client device 110 includes input/output (I/O) components 150 thatmay include a display screen (e.g., a touchscreen or a screen along witha keyboard and/or buttons for inputting user selections) 152. During useof the device 110, the image capture module or app 130 may operate togenerate and display image capture input/prompt screen 156 that isdisplayed on the device screen 152. For example, the user may initiatethe app 130 via the screen 152, and, in response, the app 130 maygenerate an input screen 156 requesting the user to select an item to befilmed in an image capture process (as shown at 162 in the memory 160).

The menu or list in the screen 156 that can be selected by the user maybe chosen by the app 130 based on the present device location (as shownin memory 160) as may be determined from data from the GPS sensor 144(e.g., what live events are known to the device's app 130 (via lookup tomemory 160 and/or by communications over a wireless communicationsnetwork (not shown in FIG. 1) to an image capture server providing liveevent schedules for all or subset of possible geographical locations ofthe device 110) that may be occurring nearby to the present devicelocation 170 within a predefined time period relative to the presenttime 174 (such as within the next 30 minutes or next 60 minutes)?). Inother cases, the module or app 130 may choose the targeted itemselection 162 for the user based on the present location 170 of thedevice 110 in combination with the present time 174 (e.g., when there isonly one live event/show at the venue where the device 110 is located,the module 130 may “assume” that this is the event to be recorded withthe camera 114).

The module or app 130 may then determine the location (e.g.,geographical or 3D coordinates) of the targeted item and store this inmemory 160 as shown at 164. The module or app 130 may retrieve thislocation from memory 160 or from an online source using the targeteditem selected (item ID) 162, and this location information 164 mayinclude a FOV where the device 110 and its camera 114 can be located forobtaining a good image 180. This information 164 can then be used by themodule/app 130 to provide feedback to the user of device 110 withfeedback mechanism 140 to properly orient the lens 116, e.g., based onthe device's present location 170 and the FOV/location for the targeteditem 164 the user may be able to simply change the pitch or azimuth ofthe camera 114 or lens 116 or may actually need to move to a newlocation 170 to be in the FOV (e.g., an audio output may instruct theuser via feedback mechanism 140 to move “X feet” in a particulardirection).

The image capture module 130 may also act to present an image captureinput screen 156 that requests the user to input or select from a set ofchoices defining start and end times for capturing the image 180. Asshown, a start and end time 166 is retrieved by the module 130 for theselected target item 162 (e.g., when does a parade or other live eventstart and how long is it scheduled to last?), and the screen 156 may begenerated or modified to ask the user to define when in this time perioda still photograph should be taken (e.g., now or some preset time fromthe start time or when a particular segment in the show is occurring) ora start time and duration for a video (e.g., start video when characteris singing a song on stage and end when song is done, start video atstart time of show and continue video for all or a portion of the show,take video at times coinciding with segment of show, start videoimmediately and continue for next “X minutes,” and so on).

The client device 110 includes a number of devices or components thatallow the image capture module or app 130 to achieve eyes-free imagecapture. First, the device 110 may include one or more feedbackmechanisms 140 that can be selectively operated by the module or app 130to guide or instruct a user/operator of the device 110 to change theorientation or position of the device 110 to better orient the camera114 and its lens 116 to have a field of view (FOV) that is desirable forcapturing the still or video digital image 180. For example, the devicemay have a rumble or vibration mechanism that can be used to let theuser know when the device 110 is or, more typically, is not in theproper orientation (e.g., vibrate the housing of the device 110 when theorientation of the lens 116 has moved outside an acceptablelens-orientation range (e.g., acceptable angle range relative tohorizontal (or pitch range), acceptable azimuth or rotation anglerelative to a vertical rotation angle, and the like). As anotherexample, a digital voice may provide verbal or audio guidance to achieveproper lens orientation (e.g., “please tilt your camera further upward”,“rotate your camera further to the left (or right)”, and so on). Inother cases, a flashing green light may be used to indicate proper lensorientation and a flashing red light may indicate that the camera114/lens 116 or device 110 should be reoriented to better aim the lens116 at the targeted item.

To assist the module 130 in capturing images, the device 110 includes aclock 142 providing the present time 174, which can be used by themodule 130 in determining when to initiate image capture with camera 114and when to stop image capture (with duration being user selectable insome cases). The device 110 also includes a mechanism 144 that providesdata for determining the present device location 170, which as discussedabove may be used by the module 130 to provide location-specific choicesfor target items/live events in input screens 156, for selecting theitem 162, and/or for instructing the user to move the device 110 to anew location to capture the image 180 of the selected target item 162.

The location of the device 110 may be determined in a number of ways andeach may be used to implement the element 144. For example, a GPS sensormay be used for location determination mechanism 144 (collectingreal-time latitude, longitude, and altitude) while other implementationsof the device 110 may use WiFi triangulation, optical marker-basedlocation techniques, or Bluetooth-based wireless techniques to implementthe mechanism 144. The device location 170 may also include informationas to the height of the device 110 (or its lens 116), and the device 144may provide data on height and/or another mechanism may be included indevice 110 to allow the module 130 to determine device (or camera lens116) height such as via infrared (IR) or optical signals transmittedfrom the device 110 and received after reflection from one or morenearby surfaces or through use of the altitude from the GPS sensor 144.

The device 110 further includes one or more components collecting dataor information pertinent to the device 110 that allow the module 130 todetermine the orientation of the camera 114 and/or its lens 116. Forexample, the device 110 may include a compass 146 that collectsinformation that the module 130 (or compass-associated software) may useto determine the device azimuth 178 (e.g., an angular measurement of anoffset from a reference vector (that typically points north) as measuredbetween the reference vector and an orthogonal vector from a surface ofthe device 110 (e.g., from its screen 152) or lens 116). Also, thedevice 110 may include an accelerometer or accelerometry mechanisms 148that operate to collect data that can be used, by the module 130 orother components in device 110, to determine the device pitch 176 (e.g.,the angle from horizontal as may be measured between horizontal and anorthogonal axis of a front plane/surface of the device 110 or, more lens116). In other cases, a gyroscope may be used in place of or in additionto the compass 146 and accelerometer 148 to provide information to themodule 130 to determine the camera lens orientation 172.

FIG. 2 illustrates an image capture method (or algorithm) 200 that maybe carried out by or via a client device as described herein such asdevice 110 of FIG. 1 with its image capture module or mobile app 130.The method 200 starts at 205 such as with a server being provided on adigital communication network (e.g., the Internet as accessible in awireless manner via a WiFi link and WiFi components on a client device),and the server may be adapted to serve a mobile app in the form of animage capture module that can be downloaded onto a user's client device(e.g., onto a smartphone, tablet, or the like). For example, theoperator of an entertainment facility may provide a mobile app to assistvisitors of the entertainment facility in easily and effectivelyrecording all or select portions of one or more of their live events orshows. Since the facility operator is providing the entertainment, theywill know beforehand the location of the performance of the liveevent/show and the schedule for its performance. Further, the providerof the entertainment/live event can determine beforehand what are theuseful sets of locations for filming or recording the live event andfrom each of these locations what are useful FOVs (e.g., from thelocation “X” in our facility, a user should orient their camera in apredefined manner “Y”). This information can be used by the mobile appto help the user both select a target element for image capture and toproperly position/locate and orient their digital camera in their clientdevice to best/better record digital images. Step 205 may furtherinclude the user operating their client device (while offsite or at thelive event facility or venue) to access the server and to download theimage capture module or app onto their client device.

The method 200 continues at 210 with a user operating their clientdevice to initiate the image capture app such as by selecting an appicon on a touchscreen. In response to being initiated, the method 200may continue with the image capture app being run by the device'sprocessor and its OS. At 215, the image capture app acts to determinethe present location of the client device (e.g., with GPS data, withWiFi triangulation, with optical markers, with Bluetooth technologies,or other location-finding mechanisms on the client device). The locationof the device may then be stored in device memory for later use (in step230), and the location may be X-Y coordinates, latitude and longitudevalues, and so on useful for identifying a specific location within alive event facility or venue or other physical setting (e.g., on astreet within a city, in a national park, and so on). At 220, the imagecapture app acts to determine the present time such as by accessing adevice clock or memory storing the current time.

The method 200 continues at 230 with the image capture module or appacting to generate and present a target selection input page on theclient device's display screen. As part of step 230, the module or appmay use the determined location to look up what live events/shows orother possible target items (e.g., national monuments, picturesquenatural objects, historic buildings, and so on) are proximate to thedevice (and its user). For example, the user may have carried theirdevice into an amusement park and be presently located in a seat in atheater amphitheater, and the module or app may determine based on thecurrent time that a live show is scheduled to start in the next fewminutes or is presently being provided to an audience. In another case,the user may have carried their client device into a football stadium,and the module or app may determine that the game is about to start andthat the halftime show is scheduled to start at a preset time in thefuture. In some cases, step 230 (and step 235) is replaced with themodule or app simply selecting the target such as when only one liveevent is being presented near the current device location.

In step 230, the prompt page may simply request confirmation of the onelive event/target identified by the module or app as associated with thedetermined device location while in other cases the page may include twoor more possible events/targets for the user to choose from that areassociated with the present device location (e.g., a show may bepresently live but another show or possible target may be startingwithin a predefined time period near the present location (such aswithin the next 1 hour or the like)). At step 235, the image capturemodule monitors the client devices I/O (such as a touchscreen) for inputindicating a selection of a target item for image capture such as a liveshow or fireworks/light display that is scheduled to start in 10 minutesor a parade that has already started and is passing nearby the clientdevice present location. If no input is received, the step 235 isrepeated.

Once a target is selected for image capture at 235, the method 200continues at 240 with the image capture module acting to update theinput page or display screen to prompt the user to select a still orvideo image capture mode. At 245, the method 200 involves the module orapp monitoring the I/O (e.g., touchscreen or keypad) for a choice ofstill or video image capture, and, often, the method 200 may alsoinvolve the user selecting a start time and a stop time (or duration) inthe case of video mode. Other embodiments may involve the module or appselecting these values based on the selection of the target item such asselection of a song sung by a character in a show (song starts at “X”and lasts “Y” minutes sets start and stop) or selection of a portion ofa parade with a particular character (who is scheduled to pass by thepresent device location from time X to time Y which sets the start andstop times for image capture). Step 245 is repeated until user selectsstill or video image capture.

Once the capture mode is chosen, the method continues at 250 with themodule or app determining the present orientation of the client device(or its camera lens). As discussed above, this may involve determiningthe device's azimuth and pitch with a compass and an accelerometerprovided in the client device or orientation may be determined in otherways known to those skilled in the arts. At 255, the method 200continues with the module or app determining the height of the clientdevice (or its camera lens).

At 260, the method 200 continues with the image capture module or appusing the device's current location and data on the targeted item (suchas a proper FOV for a live event for the current location) to determinean acceptable range of client device locations, orientations, andheights to properly record an image of the user-selected or targeteditem. This may involve answering the questions: (1) is the client devicein an acceptable location to capture a quality image of the targetedelement? and (2) if “yes,” is the client device at a proper height andorientation to have a useful FOV for image capture of the targeted item?

If the answer is “no” to either of these questions, the method 200 maycontinue at 265 with the image capture module or app acting to operatefeedback mechanisms of the client device to prompt the user to modify atleast one of the device location, device height, and device orientation.For example, the user's location may be near an event venue but be toofar away to capture quality images, and the client device may beoperated to instruct the user to move closer to the event venue (e.g.,move within “X” feet of a stage presented by text or audio output or thelike). In some cases, an acceptable range is device dependent with somedigital cameras and/or lenses being designed for larger ranges. In othercases, the module or app may recognize that the client device is closeenough (or within an acceptable range) but the device location may causethe user's view and their client camera to have one or moresight-blocking structures between their location and the live event ortargeted item. In such a case, the module or app may urge the user (suchas with text or other visual output, vibration, audio, or other feedbackdevices) to change their location to better avoid the blockingstructures.

In another example of step 265, the user may be holding their device toolow to provide a good sight line to the targeted item from the presentlocation, and the prompting may be to raise their device to a greaterheight (such as with vibration, with audio output from speakers, throughvisual feedback colors, text, flashing lights/arrows, and so on). Inother cases, the orientation may not provide a good FOV for the targeteditem from the present location and the feedback may be provided toencourage the user to reorient their device and its built in camera(camera lens) such as to change the pitch and/or azimuth withrumble/vibration feedback, with audio feedback, with visual feedback,and the like. The visual feedback provided at 265 may be to cause theentire screen to go green (or another color) when the camera is properlyoriented and to go still or flashing red (or another color) when thecamera's orientation (or height or location) is not desirable (e.g., notproviding an acceptable FOV for the targeted item from the presentdevice location).

When the device is in correct position/orientation to get a “good” FOVat 260, the method 200 continues at 270 with the image capture moduleoperating the camera to record a still or video image (based on theselected mode) of the targeted item. In some cases, though, imagecapture at 270 will start whenever the start time is reached rather thanwaiting for proper orientation/positioning of the client device to avoidmissing a portion of a live event. Then, the method 200 continues at 275with a determination of whether or not image capture is complete. Theanswer will be “yes” if the mode is “still” and the method 200 will thenend at 290 with the captured image being stored in device memory.

If in video mode, the answer at step 275 will be based on adetermination of whether or not a capture duration has elapsed. If notcomplete, the method 200 continues at 280 with a determination of thepresent orientation (and, optionally, height and/or location) of theclient device. Then, at 284, the method 200 includes determining whetherthe camera or client device is in the proper orientation. If “yes,” themethod 200 continues at 275. If “no,” the method 200 continues at 288with the image capture module or app acting to again prompt the user viafeedback mechanisms to move the client device (e.g., adjust the pitch orazimuth to have the line of sight of the camera or its lens fall withinthe acceptable FOV for the present location and the selected target itemfor the image capture process 200).

FIGS. 3-6 illustrate a client device 320 that is configured according tothe present description with image capture software (or a mobile app) asit is being used by an operator or user 305 to capture or record adigital image of a user-selected target (or targeted item) in a guidedbut eyes-free manner. In FIG. 3, a user 305 has carried their clientdevice 320 into a space 300 such as an entertainment facility or venue,an outdoor space with natural objects of beauty, or any other locale orspace where it may be desirable to capture images using the methodstaught herein. In this example, a set or stage 310 is provided nearby tothe present location of the client device 320. The set or stage 310 maybe any structural or bounded subset of the target item space 300 uponwhich a live event or other targeted item may be located. In thisexample, the stage 310 has an upper surface 312 upon which a number ofperformers (e.g., actors, dancers, singers, and so on) 315 are locatedduring a live event (e.g., a show). A front edge or end 314 of the stage310 may be used to determine a distance, d, from a front face 322 of theclient device 320 and the stage 310 when the user-selected target itemis a show involving the performers 315 on the surface 312 of the stage310 (but other portions of the set/stage 310 may be used to achieve adesired image capture).

During use of the device or during motion capture processes involvingthe device 320, the device 320 or its image capture software may act todetermine the present location (e.g., geographic coordinates) of thedevice 320 in the space 300. With this present location known, the imagecapture software can function to look up possible target items for thedevice 320 at the present time (or within a predefined time period aboutthe present time such as up to about 1 hour before or after the presenttime). As shown in FIG. 5, the client device 320 includes a body 540with a display screen 544 on a face (or front surface) 322. The imagecapture software may act to generate an image capture page 520 that isdisplayed in the screen 544, and this page 520 may provide text orimagery prompting the user of the device 320 to select a target item forfilming/image capture. In this example, the client device 320 wasdetermined to have a location that is associated with presenting a liveevent (i.e., a “Castle Show”) on the stage 310 in space 300. Thisdetermination may involve a determination that the distance, d, iswithin a preset image capture range (e.g., within a typical digitalcapture range (e.g., 50 to 300 feet or the more depending on the liveevent/target event and values selected for average distances for whichdigital cameras are useful in capturing images) for recording a still orvideo image of a show on the stage 310).

As shown in FIG. 5, the screen 544 is operated to display textindicating that the user has initiated the image capture app and the apphas determined that the user can choose to capture a particular targetitem (i.e., “It looks like you want to capture: Castle Show”). Only onetarget item is shown in this example, but the image capture module maylist two or more possible targets in some cases. The screen 544 has alsobeen used to display selection buttons 552, 554 asking the user 305 toprovide input confirming or selecting the targeted item for imagecapture by the client device 320.

In FIG. 6, the device 320 is shown after the user 305 hasconfirmed/selected a particular target item as shown by text/images 680in the screen 544. Further, the screen 544 is updated to include awindow 682 indicating the present view of the targeted item 684 (e.g.,the present state of the stage 310 as viewed through the lens of thecamera of the client device 320). The image capture software has alsoupdated the page/display on screen 544 to prompt the user 305 for inputwith buttons 690, 692 as to whether the image capture should beperformed in still or video mode. A start time and/or end time (orduration) may also be requested in the screen 544 as part of thefunction/step shown in FIG. 6.

In response to receiving the still or video selection input in FIG. 6,the image capture module of device 320 may act to determine anorientation of the client device 320 such as its face 322 relative tothe targeted item (e.g., a show/live event on stage 310). Note, the userinput may also indicate whether the front or rear camera (or front orrear lens) is being used by the operator to capture the image, and theorientation would be determined for the proper face of the device. Forexample, the azimuth of the face 322 may be found by the image captureapp, and this information may be used to determine whether the device'sFOV (or viewing angle) as shown in FIG. 3 is desirable to capture aquality image of the targeted item (e.g., the performers 315 on surface312 of stage 310). For example, a FOV for the stage 310 or its frontedge 314 may be determined for the present location of the device 320,and it may be determined whether or not a ray that is orthogonal to theface 322 of the device 320 falls within this FOV or more preferablybisects the FOV (is at or near the center of the FOV). If theorientation is not desirable, the image capture app may operate feedbackmechanisms in the device (e.g., the rumble, audio devices, orvisual/lighting components) to urge or guide the user 305 to rotate thedevice 320 to better orient the face 322 (or the camera lens) relativeto the targeted item in space 300. Once properly oriented (or whileorientating is being performed), the image capture module may operatethe camera of the device 320 to capture the targeted item (here “theCastle Show”) in the selected capture mode (e.g., a still shot or avideo clip).

The orientation and/or positioning determination may also include, asshown in FIG. 4, the image capture software causing the device 320 todetermine the present height, h, of the client device 320. If this isnot acceptable for capturing images at the device's present location(such as due to structure blocking the lens of the camera from seeingall of the stage 312 or the performers 315), the image capture softwaremay cause the feedback mechanisms of the device 320 to be operated toprompt the user 305 to raise (or even lower in some cases) the device320 to alter the height, h, to be within an acceptable height range atthe present device location and for the selected target item in space300.

Further, the image capture module may cause the device 320 to functionto determine the present pitch, θ_(pitch), of the client device 320,which may be measured as the angle between horizontal 470 and a ray 472extending orthogonally from the device face 322 (e.g., from a cameralens front surface (not shown)). If the pitch is not within anacceptable pitch range for the selected target item and the device'spresent location, the image capture software may activate feedbackmechanisms to urge the user 305 to tip the face 322 upward or downward(rotate the device 310 about a horizontal axis) to move the device 310(or its camera lens) into a proper pitch to better capture/record imagesof the targeted item in the space 300 from the present device location.

Although the invention has been described and illustrated with a certaindegree of particularity, it is understood that the present disclosurehas been made only by way of example, and that numerous changes in thecombination and arrangement of parts can be resorted to by those skilledin the art without departing from the spirit and scope of the invention,as hereinafter claimed.

In some embodiments, the image capture module or app is furtherconfigured to enable the user of the client device to augment thecaptured image. For example, after image capture is completed asdescribed above, the module or app may act to prompt the user to selectimage augmentation. When selected, the module or app may act todetermine what types of augmentation are available for the particularcaptured image (e.g., by accessing a web server storing augmentationlayers/images for each of a number of possible target items (e.g., liveevents put on by an entertainment facility operator or the like)). Forexample, the targeted item may be a live show presented at a particularvenue, and the augmentation may be “layers” from a set associated withthe live show. These layers may be images of actual set pieces presentat or used during the performance but in better quality or these may beimages of real or animated set pieces or other augment elements createdfor the targeted item (live show in this example). The user may selectwhich augmentation layers to apply, and these layers are combined withor applied to the captured digital image to create an augmented digitalimage.

In some cases, the user may also be able to personalize one or more ofthese augmentation elements, e.g., select colors, select characters toinclude, enter data such as their name, time of image capture, anddescriptive text (e.g., “Lightshow from New Year's Eve 2016,” “Parade atOur Favorite Amusement Park During Jenny's 10^(th) Birthday,” and soon). Note, proper augmentation may require information stored for theclient device during the image capture such as device location,distance, orientation, and height as the augmentation layers areselected or adjusted to match this device information to enhance thequality of the augmentation product (e.g., added set elements aredigitized to appear as if they were captured by the client device withits location, height, and orientation).

In the same or other embodiments, the image capture module may beadapted to assist the user to capture a particular target item thatmoves during the image capture time period or duration. For example, auser may select to capture a live event that includes a plurality ofperformers. The image capture module may be configured to determine thelist of performers (e.g., characters from a movie, athletes on ateam(s), singers/dancers, and so on) in the live event and present asubset that can be tracked for better or specific image capture duringthe image capture period (duration of a video). The user may select oneof these performers, and the image capture module may then act todetermine the location of the selected performer or particular targetitem within the live event throughout the image capture period. Thisinformation is then used to provide feedback to the user via the clientdevice's feedback mechanisms to urge or assist them in moving theirclient device to follow the particular target item even as the targetmoves relative to the user, and this moving of the client device mayinvolve simply altering the azimuth (rotating the device's face orcamera lens about a vertical rotation axis) or may also involve changingthe device's pitch (such as when the performer moves up or down stage orchanges their height relative to the user's client device).

In the same or yet a further embodiment, the image capture module or appmay be configured to activate or operate feedback mechanisms in theclient device that cause the client device to move toward the desired oracceptable orientation for the particular client device location and thetargeted item for image capture. For example, the image capture modulemay function to determine that the client device needs to have a pitchthat differs from the present pitch, and the module may act to force thedevice to move in the user's hands to have a pitch within the acceptablepitch range (a similar process can be followed to adjust the azimuth).Hence, the body of the client device may be moved by the image capturemodule or app via onboard movement mechanisms to focus the camera towardthe targeted item in an automated manner or to keep the device's camerain focus if the user should move after focus is achieved during theimage capture period.

Several means are available to implement the systems (such as device 110of FIG. 1) and methods discussed in this specification. These meansinclude, but are not limited to, digital computer systems,microprocessors, application-specific integrated circuits (ASIC),general purpose computers, programmable controllers and fieldprogrammable gate arrays (FPGAs), all of which may be genericallyreferred to herein as “processors.” For example, in one embodiment,signal processing may be incorporated by an FPGA or an ASIC, oralternatively by an embedded or discrete processor. Therefore, otherembodiments include program instructions resident on computer readablemedia which when implemented by such means enable them to implementvarious embodiments. Computer readable media include any form of anon-transient physical computer memory device. Examples of such aphysical computer memory device include, but are not limited to, punchcards, magnetic disks or tapes, optical data storage systems, flash readonly memory (ROM), non-volatile ROM, programmable ROM (PROM),erasable-programmable ROM (E-PROM), random access memory (RAM), or anyother form of permanent, semi-permanent, or temporary memory storagesystem or device. Program instructions include, but are not limited to,computer-executable instructions executed by computer system processorsand hardware description languages such as Very High Speed IntegratedCircuit (VHSIC) Hardware Description Language (VHDL).

I claim:
 1. A client device adapted for enhanced image capture,comprising: a processor; memory accessible by the processor; a camerawith a lens, the camera capturing a digital image via the lens andstoring the captured digital image in the memory; a body containing theprocessor, the memory, and the camera; and an image capture applicationrun by the processor to select a target item to be included in thecaptured digital image, wherein the image capture application causes theprocessor to operate one or more feedback mechanisms to prompt anoperator of the client device to move the body to move the lens of thecamera, before or during the capturing of the digital image, from afirst orientation to a second orientation relative to the target item,wherein the image capture application determines the first orientationand the second orientation using orientation sensing components in theclient device, wherein the first orientation comprises a first pitch anda first azimuth and the second orientation comprises a second pitch anda second azimuth, wherein the image capture application further acts todetermine a present location of the client device, wherein, when thesecond orientation is within a range of acceptable orientations forcapturing an image of the target item from the present location, theimage capture application automatically initiates the capturing of thedigital image, and wherein the image capture application further acts todetermine a first height of the lens of the camera, to compare the firstheight of the camera to an acceptable range of heights for capturing theimage of the target item at the present location of the client device,and when the first height is outside of the acceptable range of heightsto operate the one or more feedback mechanisms to prompt the operator tomove the camera from the first height to a second height within theacceptable range of heights.
 2. The client device of claim 1, whereinthe second orientation is selected to provide the camera with a field ofview of the target item from the present location.
 3. The client deviceof claim 1, wherein the feedback mechanisms include a vibrationmechanism selectively operable until the lens of the camera is in thesecond orientation.
 4. The client device of claim 1, wherein thefeedback mechanisms include a mechanism adapted to move the body of theclient device within a hand of the operator to automatically move thelens of the camera from the first orientation to the second orientation.5. The client device of claim 1, wherein the feedback mechanisms includea mechanism prompting the operator to move the client device byproviding audio cues.
 6. The client device of claim 1, wherein thetarget item is selected by the image capture application based on adetermination of the present location of the client device.
 7. Theclient device of claim 1, wherein the image capture application acts tolookup a set of potential targets local to the present location, and todisplay the set of potential targets on a display screen of the clientdevice to prompt the operator to indicate the target item from the setof potential targets.
 8. A client device adapted for enhanced imagecapture, comprising: a processor; memory accessible by the processor; acamera with a lens, the camera capturing a digital image via the lensand storing the captured digital image in the memory; a body containingthe processor, the memory, and the camera; and an image captureapplication run by the processor to select a target item to be includedin the captured digital image, wherein the image capture applicationcauses the processor to operate one or more feedback mechanisms toprompt an operator of the client device to move the body to move thelens of the camera, before or during the capturing of the digital image,from a first orientation to a second orientation relative to the targetitem, wherein the image capture application responds to user input toretrieve from the memory and to augment, and then store in the memory,the captured digital image previously captured during the capturing bythe camera using augmentation layers selected or modified based on atleast one of an orientation of the lens of the camera relative to thetarget item, a distance between the lens of the camera and the targetitem, and a height of the lens of the camera during the capturing of thedigital image.
 9. A device for recording digital images, comprising: adigital camera; a location sensor sensing a present location of thedevice; a display screen; and an image capture module retrieving a setof potential targets for imaging from the present location, generating aprompt page displayed on the display screen, receiving user input, viathe prompt page on the display screen, selecting one of the potentialtargets for use as a target item, determining a range of orientationsfor the digital camera for use during image capture for the target itemand for the present location, prompting a user to move the digitalcamera into an orientation within the range of orientations, andautomatically initiating operation of the digital camera to capture adigital image of the target item without additional user input, whereinthe receiving of the user input is performed prior to focusing of thedigital camera on the target item, and wherein the image capture moduleresponds to additional user input to retrieve from the memory and toaugment, and then store in the memory, the captured digital imagepreviously captured during the capturing by the digital camera usingaugmentation layers selected or modified based on at least one of anorientation of lens of the digital camera relative to the target item, adistance between the lens of the digital camera and the target item, anda height of the lens of the digital camera during the capturing of thedigital image.
 10. The device of claim 9, wherein the prompting of theuser to move the digital camera comprises providing tactile or audiocues to the user via feedback mechanisms of the device.
 11. The deviceof claim 9, wherein the prompting of the user is performed before andduring the operation of the digital camera to capture the digital image,whereby focus assist is provided during image capturing with the digitalcamera.
 12. The device of claim 9, wherein the image capture modulefurther acts to augment the captured digital images with one or moreaugmentation layers selected or modified to match the present locationof the device and the orientation of the digital camera during the imagecapture.
 13. An image capture method, comprising: determining a locationof a digital camera; based on the location, selecting a target for imagecapture; determining a first orientation of the digital camera; andbased on the target for image capture, providing feedback to move thedigital camera from the first orientation to a second orientation withina range of orientations for image capture of the target from thelocation, wherein the providing feedback is performed during operationsof the digital camera to capture of a video image of the target and theproviding feedback comprises providing tactile or audio cues to the uservia feedback mechanisms of the digital camera, and wherein the methodfurther comprises responding to user input to retrieve from memory andto augment and then store in the memory, the video image previouslycaptured using augmentation layers selected or modified based on atleast one of an orientation of a lens of the digital camera relative tothe target, a distance between the lens of the digital camera and thetarget, and a height of the lens of the digital camera during theoperations of the digital image to capture the video image of thetarget.
 14. The method of claim 13, wherein the target for image capturecomprises a portion of a venue for a live event and wherein the methodfurther comprises providing feedback to move the digital camera from thesecond orientation to a third orientation to track movement of auser-selected element of the live event during an image capture processfor the target.
 15. The method of claim 13, further comprisingdetermining a first height of the digital camera at the location andbased on the location and the target for image capture providingfeedback to move the digital camera from the first height to a secondheight within a range of heights for image capture of the target fromthe location.
 16. The method of claim 13, wherein the first orientationcomprises a pitch of the lens of the digital camera and an azimuth ofthe lens of the digital camera.